Evaluating User Preferences in Machine Translation Using Conjoint Analysis
نویسندگان
چکیده
In spite of much ongoing research on machine translation evaluation there is little quantitative work that directly measures users’ intuitive or emotional preferences regarding different types of machine translation errors. However, the elicitation and modeling of user preferences is an important prerequisite for future research on user adaptation and customization of machine translation engines. In this paper we explore the use of conjoint analysis as a formal quantitative framework to gain insight into users’ relative preferences for different translation error types. Using English-Spanish as the translation direction we conduct a crowd-sourced conjoint analysis study and obtain utility values for individual error types. Our results indicate that word order errors are clearly the most dispreferred error type, followed by word sense, morphological, and function word errors.
منابع مشابه
Investigation of Customer Priorities for Machine Made Carpet Through Conjoint and Cluster Analysis (Case Study in Yazd, Iran)
The machine made carpet industry is one of the main and most famous industries in Iran and especially in the city of Yazd. However there is little information about customer preferences for different attributes of this product. In this article we tried to estimate the relative importance of the main attributes affecting customer desire for purchasing machine made carpet and the utility values f...
متن کاملA Machine Learning Approach to Conjoint Analysis
Choice-based conjoint analysis builds models of consumer preferences over products with answers gathered in questionnaires. Our main goal is to bring tools from the machine learning community to solve this problem more efficiently. Thus, we propose two algorithms to quickly and accurately estimate consumer preferences.
متن کاملThe Correlation of Machine Translation Evaluation Metrics with Human Judgement on Persian Language
Machine Translation Evaluation Metrics (MTEMs) are the central core of Machine Translation (MT) engines as they are developed based on frequent evaluation. Although MTEMs are widespread today, their validity and quality for many languages is still under question. The aim of this research study was to examine the validity and assess the quality of MTEMs from Lexical Similarity set on machine tra...
متن کاملAdaptive Conjoint Analysis. Training Data: Knowledge or Beliefs?
The foundational model of conjoint analysis is to model consumer purchase preferences by means of utility functions. Analysts run surveys and interviews to obtain a basic set of training data, typically user preferences on which the utility function is mapped. The utility theory trust the training data as knowledge while there is large literature emphasizing that users preference may change, ma...
متن کامل